chore(ci): fix coverage when modifying existing ported tests #2013
+14
−2
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
🗒️ Description
The coverage CI doesn't handle cases where we modify existing ported tests correctly. This PR resolves that issue.
Problem
When modifying an existing test file that contains
@pytest.mark.ported_from
markers from past porting, the coverage CI would:any_ported=true
BASE_TESTS
for coverage comparisonBASE_TESTS
index filesSolution
This fix operates on the principle that once a test is ported, the Python version becomes the source of truth. We distinguish between:
@pytest.mark.ported_from
markers in the git diff, means we only compare against original JSON from ethereum/tests.@pytest.mark.ported_from
markers in git diff, means we compare against the previous Python version.Future Considerations
The coverage script is not capable of handling cases where new ported tests are added to an existing file that modifies existing tests. The recommendation for porting any new test from
ethereum/tests
is to a) add it to a new file but b) at a minimum do not modify any existing tests if added to an existing file.🔗 Related Issues or PRs
#2002
✅ Checklist
tox
checks to avoid unnecessary CI fails, see also Code Standards and Enabling Pre-commit Checks:uvx --with=tox-uv tox -e lint,typecheck,spellcheck,markdownlint
type(scope):
.